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Abstract 

The Boltzmann entropy is true in the case of equal probability of all mi- 

crostates of a system. In the opposite case it should be averaged over all microstates 
that gives rise to the Boltzmann-Shannon entropy (BSE). Maximum entropy princi- 
ple (MEP) for the BSE leads to the Gibbs canonical distribution that is incompatible 
with power-low distributions typical for complex system. This brings up the ques- 
tion: Does the maximum of BSE correspond to an equilibrium (or steady) state of 
the complex system? Indeed, the equilibrium state of a thermodynamic system which 
exchange heat with a thermostat corresponds to maximum of Helmholtz free energy 
rather than to maximum of average energy, that is internal energy U . Following 
derivation of Helmholtz free energy the Renyi entropy is derived as a cumulant av- 
erage of the Boltzmann entropy for systems which exchange an entropy with the 
thermostat. The application of MEP to the Renyi entropy gives rise to the Renyi dis- 
tribution for an isolated system. It is investigated for a particular case of a power-law 
Hamiltonian. Both Lagrange parameters, a and f3 can be eliminated. It is found 
that (5 does not depend on a Renyi parameter q and can be expressed in terms of 
an exponent k of the power-law Hamiltonian and U . The Renyi entropy for the 
resulting Renyi distribution reaches its maximal value at q = 1/(1 + n) that can be 
considered as the most probable value of q when we have no additional information 
on behavior of the stochastic process. The Renyi distribution for such q becomes a 
power-law distribution with the exponent — (k + 1) . Such a picture corresponds to 
some observed phenomena in complex systems. 
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entropy, escort distribution, power-law distribution. 



1 Introduction 

The well-known Boltzmann formula defines a statistical entropy, as a logarithm of number 
of states W attainable for the system 

S {B) =\nW (1) 

Here and below the entropy is written as dimensionless value without the Boltzmann con- 
stant ks ■ 

This definition is valid not only for physical systems but for much more wide class of 
social, biological, communication and other systems described with the use of statistical 
approach. The only but decisive restriction on the validity of this equation is the condition 
that all W states of the system have equal probabilities (such systems are described in 
statistical physics by a microcanonical ensemble). It means that probabilities pi = pw = 
1/W (for all i = 1,2, W ) that permits to rewrite the Boltzmann formula (1) as 

S {B) = -\np w . (2) 
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When the probabilities pi are not equal we can introduce an ensemble of microcanonical 
subsystems in such a manner that all Wi states of the i -th subsystem have equal prob- 

( Fi) 

abilities Pi and its Boltzmann entropy is = — lnpj . The simple averaging of the 

Boltzmann entropy SI leads to the Gibbs-Shannon entropy 

S (G) = {S (B) )p ^_J2p i lnp l . (3) 

i 

Just such derivation of is used in some textbooks (see, e. g. (HE])- This entropy 

is generally accepted in statistical thermodynamics and conventional in a communication 
theory. 

According to maximum information principle (MEP) developed by Jaynes [3] for a 
Gibbs-Shannon statistics an equilibrium distribution of probabilities p = {pi} must pro- 
vide maximum of the Gibbs-Shannon information entropy S^ B \p) = —ksYsi Pi^ n Pi upon 
additional conditions of normalization X)i Pi = 1 an d a fixed average energy 

U = (H) p = Y, H iPi, ( 4 ) 

i 

where H = J2iHi is the Hamiltonian of the system. 

Then, the distribution {p^} is determined from the extremum of the functional 

W WW 

L g(p) = ~ J2 Pi ln Pi ~ a Pi ~ Po X! H &i> ( 5 ) 

i i i 

where «o and /3q are the Lagrange multipliers. Its extremum is ensured by the Gibbs 
canonical distribution 

pP = Z?e-** (6) 

in which /3 is determined by condition of correspondence between Gibbs thermostatistics 
and classical thermodynamics as (3q = 1//cbT where T is the thermodynamic tempera- 
ture. 

However, when investigating complex physical systems (for example, fractal and self- 
organizing structures, turbulence) and a variety of social and biological systems, it appears 
that the Gibbs distribution does not correspond to observable phenomena. In particular, 
it is not compatible with a power-law distribution that is typical [4J for such systems. 
Introducing of additional restrictions to a sought distribution in the form of conditions of 
true average values (X^ m ') p of some physical parameters of the system gives rise 

to a generalized Gibbs distribution with additional terms in the exponent (6) but does not 
change its exponential form. 

2 Helmholtz free energy and Renyi entropy 

There is no doubts about the MEP as itself, because of it is a kind of "a maximum honesty 
principle" according to which we demand a maximal uncertainty from the distribution apart 
from true description of prescribed averages. In the opposite case we risk to introduce a 
false information into the description of the system. 

Thus, it only remains for us to throw doubt on the information entropy form. To seek 
out a direction of modification of the Gibbs-Shannon entropy we consider first extremal 
properties of an equilibrium state in thermodynamics. 
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A direct calculation of the average energy of a system gives the internal energy (4), its 
extremum is characteristic of an equilibrium state of rest for a mechanical system, other 
than a thermodynamic system that can change heat with a thermostat. An equilibrium 
state of the latter system is characterized by extremum of the Helmholtz free energy F . 
To derive it statistically from the Hamiltonian without use of thermodynamics we introduce 
generating function ^introduce generating function for the random value H = —J2iHi 

^(A) = E^% (7) 

i 

where A is the arbitrary constant, and construct a cumulant generating function 

tf H (A) = ln$ H (A) (8) 

that becomes the Helmholtz free energy F when devided by A that is chosen as A = 
-l/k B T . 

Now we return to the problem of a generalized entropy for open complex systems. 
Exchange by both energy and entropy is characteristic for them. As an illustration, there 
is a description by Kadomtsev |H1 of self-organized structure in a plasma sphere: "The 
entropy is being born continuously within the sphere and flowing out into surroundings. 
If the entropy flow had been blocked, the plasma would 'die'. It is necessary to remove 
continuously 'slag' of newly produced entropy". 

That is the reason why the Gibbs-Shannon entropy, derived by the simple averaging 
of the Boltzmann entropy can not be a function of which extremum characterizes a steady 
state of a complex system which exchange entropy with surroundings, just as the minimum 
of the internal energy does not characterize an equilibrium state of the thermodynamic 
system being in heat contact with a heat bath. 

An effort may be made to find a "free entropy" of a sort by the same way that was used 
above for derivation of the Helmholtz free energy. The generating function is introduced 
as 

$ 5 (A) = £e A5 ' S) (9) 

i 

Then the cumulant generating function is 

tf s (A)=]n$ s (A)=ln$>r A - (10) 

i 

To obtain the desired generalization of the entropy we are to find a A -dependent numer- 
ical coefficient which ensures a limiting pass of the new entropy into the Gibbs-Shannon 
entropy. Such the coefficient is (A + 1) . Indeed, the new A -family of entropies 

S(\) = T ^\nJ2p7 X . (11) 

includes the Gibbs-Shannon entropy as a particular case when A — > — 1 . 

Thus, it has appeared that the desired "free entropy" coincides with the known Renyi 
entropy [Zj. It is conventional to write it with the parameter q = —A in the form 

S {R) (P) = ^ In Erf (12) 
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The same result can be obtained with the use of a Kolmogorov-Nagumo [HI [HI generalized 
averages of the form 

(x)* = W E^(^)) (13) 

where 4>(x) is an arbitrary continuous and strictly monotonic function and is the 

inverse function. It had been just this kind of average that was used by Renyi [TH] to define 
his new entropy as a generalized average of the Boltzmann entropy (1). As a result of choice 
of the Kolmogorov-Nagumo function in the form (f)(x) = &- 1 ~ q ' x he obtained (12). Such 
the choice of <p(x) appears accidental until it is not pointed that the same exponential 
function of the Hamiltonian provides derivation of the free energy that is extremal at an 
equilibrium state of a thermodynamic system which exchange heat with a heat bath. This 
fact permits to suppose that the Renyi entropy derived in the same manner is extremal 
at a steady state of a complex system which exchange entropy with its surroundings very 
actively. 

Different properties of the Renyi entropy are discussed in particular in Refs. [T| fTT| fT2] . 
It is positive ( > ), convex at q < 1 , passes into the Gibbs-Shannon entropy 
lim^i S( R ' = S G and into Boltzmann entropy (1) for any q in the case of equally probable 
distribution p . 

In the case of \1 — J2iPi\ ^ 1 (which, in view of normalization of the distribution 
{pi} , corresponds to the condition |1 — q\ <C 1 ), one can restrict oneself to the linear 
term of logarithm expansion in the expression for S^ R '(p) over this difference, and S^ R '(p) 
changes to the Tsallis entropy [EU 

S {T \p) = -^-(l -Y.PI). (14) 

The logarithm linearization results in the entropy becoming nonextensive, that is, S(WiW2) ^ 
S(Wi)S(W 2 ) . This property is widely used by Tsallis and by the international scientific 
school that has developed around him for the investigation of diverse nonextensive systems 
(see web site [HI). I n so doing, the above-identified restriction |1 — q\ <C 1 is disregarded. 
As a result of nonextensivity the Tsallis entropy incompatible with the Boltzmann entropy 
(1) because of the latter was derived by Plank in such the form just from the extensivity 
condition S(W 1 W 2 ) = S(W 1 )S{W 2 ) . 

3 MEP for Renyi entropy 

If the Renyi entropy is used in MEP instead of the Gibbs-Shannon entropy, the equilibrium 
distribution is to be looked for from the maximum of the functional 

I w w w 

Lr{p) = —— Pi ~ «E Pi ~P J2 H iPii ( 15 ) 

9 i i i 

where a and (3 are Lagrange multipliers. It can be noticed that L R {p) passes to L G {p) 
in the q — > 1 limit. 

We equate a functional derivative of Lr{j>) to zero, then 



SL R (p) q p 
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8pi l-?EiP 



a- f3Hi = 0. (16) 



4 



To eliminate the parameter a we can multiply this equation by pi and sum up over i , 
taking into account the normalization condition J2i Pi — 1 ■ Then we get 

a = -«--pU (17) 
1 - q 

and 

(1 - E Pi J , Aifj = Hi-U. (18) 
Using once more the condition Pi = 1 we get 

EpJ=(E(i-^Aff i )^J 

and, finally, we get [13 [THJ the Renyi distribution 



Pi = P., 



\ R) = Z^fl-^AHi)" 1 (19) 
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Zr = E 1-/5^— Afl< • (20) 



When applying MEP to the Tsallis entropy it is necessary to take into account that all 
average values in the last version of nonextensive thermostatistics [E| are calculated with 
the use of an escort distribution Pi = pf/J2iPi that contradicts to the main principles 
of probability description. Indeed, at q > 1 (q < 1 ), the importance of Pi with the 
maximal (minimal) values increases. In view of this, it is evident that use of the escort 
distribution in statistical thermodynamics does not lead to true average values of dynamical 
variables. Nevertheless, the additional condition of a fixed mean energy in nonextensive 
thermostatistics is written as U = (H) es = J^i^iPi and the escort Tsallis distribution is 
found in the form 

Pf s) = Z?l (1 - /T(l - q')AH t )^7 (21) 

where /3* = /3/J2iPi an d P is the Lagrange multiplier. 

It should be noticed that both Renyi and escort Tsallis distributions, Eqs. (19) and 
(21), are identical if q' — 1/q . In reality, in this case 

, q — 1 q' 1 , , 

W = - — , t^ = — 7 22 

q 1 — q' q — 1 

and P* is determined by the same second additional condition (4) of MEP as well as (3 . 

Thus, not always justified linearization of the logarithm in the Renyi entropy and ques- 
tionable use of the escort distribution leads to the same Renyi distribution if q and q' are 
considered as free parameters. 

Therefore, numerous works (see [l4j) confirming correspondence of the Tsallis escort 
distribution (with fitted q' ) with distributions in complex physical, biological, social and 
other systems count rather in favor more justified Renyi entropy than nonextensive Tsallis 
entropy. 

When q — > 1 the distribution {pl^} becomes the Gibbs canonical distribution and 
P/q ~ y A) = 1/ksTo . Such behavior is not enough for unique determination of P , as in 
general, it may be an arbitrary function P(q) which becomes Po in the limit q — ► 1 . 
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To find an explicit form of (3 , we return to the additional condition of the pre-assigned 
average energy and substitute there the Renyi distribution (19). Then we obtain the integral 
equation for (3 

w 

U = Y,HiP?\ (23) 

i 

where U is considered as a known value. This equation was solved (.16] for a particular 
case of a power-law dependence of the Hamiltonian on a parameter x 

Hi = Cx R % . ( 24 ) 

This type of the Hamiltonian corresponds to an ideal gas model in the Boltzmann-Gibbs 
thermostatistics and it seems reasonable to say that it may be useful in construction of 
thermostatistics of complex systems. Moreover, in most social, biological and humanitarian 
sciences the system variable x can be considered (with n = 1 ) as a kind of the Hamiltonian 
(e.g. the size of population of a country, effort of a word pronouncing and understanding, 
bank capital, number of scientific publications of an author, size of an animal etc.). 

For the power-law Hamiltonian the convergence condition for the sum in Eq. (23) in 
the limiting case W — > oo is 

q > q-min = 1/(1 + «) (25) 
As a result of solution of equation (23) the parameter f3 is found as 

p= w (26) 

Independence of this relation from q means that it is true, in particular, for the limit case 
q — 1 where the Gibbs distribution takes a place and, therefore, 

P = Pq = 1/JfeflT for all q. (27) 

When H = p 2 /2m (that is, k = 2 ) we get from (26) and (27) that U = \k B T , as 
would be expected for one-dimensional ideal gas. 

The Lagrange parameter (3 can be eliminated from the Renyi distribution (19) with 
the use of Eq. (26) and we have, alternatively, 

p W = z -i( 1 _g^( CuX «_ 1 )\ 9 ~\ Hi = Cxl C U = C/U. (28) 



The problem to be solved for a unique definition of the Renyi distribution is determi- 
nation of a value of the Renyi parameter q . This will be the subject of the next section. 



4 The most probable value of the Renyi parameter 

An excellent example of solution of this problem for a physical non-Gibbsian system was 
presented by Wilk and Wlodarczyk [TH]. They took into consideration fluctuations of both 
energy and temperature of a minor part of a large equilibrium system. This is a radical 
difference of their approach from the traditional Gibbs method in which temperature is a 
constant value characterizing the thermostat. As a result, their approach led (see [19J) to 
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Figure 1: The entropies S^ R '\p^ R \q, «)] (top) and S^\p( R \q, «)] (bottom) for the power- 
law Hamiltonian with the exponent K within the range 3 > K > 0.5 and g > 1/(1 + k) . 



the Renyi distribution with the parameter q expressed via heat capacity Cy of the minor 
subsystem 

9 = ^7^- (29) 

The approach by Wilk and Wlodarczyk was advanced by Beck [20] and Beck and Cohen 
[2*T] who offered for it a new apt term "superstatistics". In the frame of superstatistics, the 
parameter q is defined by physical properties of a system which can exchange energy and 
heat with a thermostat. As a result, q ^ 1 but \q — 1| -C 1 , because of exchange entropy 
is not taken into account by superstatistics. 

In general, when we have no information about nature of a stochastic process the pa- 
rameter q is considered as a free parameter. So, the proposed further extension of MEP 
consists in looking for a maximum of the Renyi entropy in a space of the Renyi distributions 
with different values of q . 

In reality, maximum of the RE is ensured by the Renyi distribution function (28) 
for any fixed q fulfilling the inequality (25). The next step consists of substitution of 
the Renyi distribution p^ R \q,n) into the definition of the Renyi entropy, Eq. (13), and 
variation of the q -parameter. The resultant picture of S^\p^ R \q, k)] as a function of q 
is illustrated in Fig. 1 (left). It is seen that S^\p^ R \q, k)] attains its maximum at the 
minimal possible value q = q min . For q < q min , the series (23) diverges and, therefore, 
the Renyi distribution does not determine the average value U = (H) p , that is a violation 
of the second condition of MEP. 

The similar procedure is applied to the Gibbs-Shannon entropy S^ B \p) , as well. Substi- 
tuting there p = p( R \q, k) we get the q -dependent function S^[p^ R \q, k)} illustrated in 
Fig. 1 (right). As would be expected, the Gibbs-Shannon entropy S^\p^ R \q, k)} attains 
its maximum value at q = 1 where p( R '(q,K) becomes the Gibbs canonical distribution. 

Thus, it is found that the maximum of the Renyi entropy is realized at q = q m i n and 
it is just the value of the Renyi parameter that should be used if we have no additional 
information on behavior of the stochastic process under consideration. 

Recall that the Renyi entropy was derived above as a functional which attain its max- 
imum value at the equilibrium state (or steady state) of a complex system just as the 
Helmholtz free energy for a thermodynamic system. (The kind of extremum, that is, max- 
imum or minimum is determined by the sign definition for the constant A .) Then, a 
radically important conclusion follows from comparision of these two graphs: 

In contrast to the Gibbs-Shannon entropy the Renyi entropy increases as a system 



complexity (departure of the q value from 1) increases that permits to explain an evolution 
of the system to self-organization from the point of view of thermo dynamical stability. 

It counts in favor of such the conclusion that a power-law distribution characteristic 
for self-organizing systems |1] is realized when the Renyi entropy is maximal. Substitution 
of q = q m i n into Eq. (28) does lead to 

p = Z- l x-^ K) (30) 

Thus, for q = q m in the Renyi distribution for a system with the power-law Hamiltonian 
becomes power-low distribution over the whole range of x . 

For a particular case of the impact fragmentation where H ~ m 2//3 the power-law 
distribution of fragments over their masses m follows from (21) as p(m) ~ m~ 5 / 3 that 
coincides with results of our previous analysis §H] and experimental observations [2T?] . 

For another particular case, k = 1 , power-low distribution is p ~ x~ 2 . Such a form 
of the Zipf-Pareto law is the most useful in social, biological and humanitarian sciences. 
The same exponent of power-low distribution was demonstrated [2H for energy spectra 
of particles from atmospheric cascades in cosmic ray physics and for distribution of users 
among the web sites |25j. 

I acknowledge fruitful discussions of the subject with A. V. Vityazev. 
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